Video communication method and robot for implementing the method

ABSTRACT

Provided are a video communication method and a robot implementing the same. The robot includes a camera configured to acquire a first video of a space for a video call, a multi-channel microphone configured to receive a sound signal output to the space, a memory storing one or more instructions, and a processor configured to execute the one or more instructions. The processor determines a first user among N users in the first video based on the sound signal received in a previous time period prior to a first time point, wherein the first user is a main user of the video call at the first time point and N is an integer greater than or equal to 2.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation Application of prior U.S. patentapplication Ser. No. 16/557,310 filed Aug. 30, 2019, which claimspriority under 35 U.S.C. § 119 to Korean Application No. 10-2019-0093717filed on Aug. 1, 2019, whose entire disclosures are hereby incorporatedby reference.

BACKGROUND 1. Field

The present invention relates to a video communication method and arobot for implementing the same.

2. Background

With the rapid development of communication-related technology, a videocall function of enabling a caller and a called party to talk whilelooking at the other party's face has been activated. Through the videocall function, a call is made while transmitting and receiving videoscaptured by cameras of a calling terminal and a called terminal.

The video call function may be performed through a robot which includesa camera, a speaker, a microphone, a communicator, and a display. Therobot may transmit a first video acquired by the robot to a counterpartrobot, receive a second video from the counterpart robot, display thesecond video on a display thereof, and make a video call using thesecond video.

There may be a case in which a video call is made by a plurality ofusers through one robot. In particular, a situation may occur in which avoice signal output from a user who does not appear on a screen of thedisplay is transmitted to a counterpart user. In this case, the qualityof the video call decreases due to the voice signal of the user who doesnot appear on the screen.

In addition, when there are two or more users on the screen, it isdifficult to determine which user is a main speaker of the video call.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments will be described in detail with reference to thefollowing drawings in which like reference numerals refer to likeelements wherein:

FIG. 1 is a view illustrating the appearance of a robot according to anembodiment of the present invention;

FIG. 2 is a block diagram illustrating a control relationship betweenmain components of a robot according to an embodiment of the presentinvention;

FIG. 3 is a flowchart of a video communication method using a robotaccording to a first embodiment of the present invention;

FIG. 4 is a diagram illustrating a concept of a voice output section anda non-voice output section of a user's voice signal according to anembodiment of the present invention;

FIG. 5 is a flowchart of a video call method using a robot according toa second embodiment of the present invention;

FIG. 6 is a view for explaining the concept of previous time periodsaccording to an embodiment of the present invention; and

FIG. 7 is a flowchart of a video communication method performed betweena robot and a counterpart robot according to a third embodiment of thepresent invention.

DETAILED DESCRIPTION

Hereinafter, embodiments of the present invention will be described indetail with reference to the accompanying drawings so that those ofordinary skill in the art can easily implement the embodiments. Thepresent invention may be embodied in many different forms and is notlimited to the embodiments set forth herein.

For clarity, parts not related to explaining the present invention areomitted herein, and the same reference numerals are assigned to the sameor like components throughout the specification. Some embodiments of thepresent invention will be described below in detail with reference toexemplary drawings. In the drawings, the same reference numerals areused to denote the same elements throughout the drawings even when theelements are shown in different drawings. In the following description,well-known functions or constructions are not described in detail whenit is determined that they would obscure the invention due tounnecessary detail.

In describing components of the present invention, the terms “first,”“second,” “A,” “B,” “(a),” “(b),” etc. may be used. These terms aremerely used herein to distinguish each component from other componentsand thus the natures, sequence, order, or number of the componentscorresponding thereto are not limited by these terms. When an element isreferred to as being “coupled to,” “combined with,” or “connected to”another element, it should be understood that the element may bedirectly coupled to, combined with, or connected to the other element ormay be coupled to, combined with, or connected to the other componentwhile another element is interposed therebetween or via another element.

For convenience of explanation, elements of the present invention may bedivided into sub-elements and explained but the elements may be includedin one device or module or one element may be divided into sub-elementsand included in a plurality of devices or modules.

Hereinafter, the term “robot,” when used herein, may refer to a machinethat automatically handles a given task or operates with its owncapabilities. In particular, a robot having a function of identifying anenvironment and determining and performing an operation with its owndecision may be referred to as an intelligent robot.

Robots may be classified into industrial robots, medical robots,household robots, military robots, etc. according to a purpose or fieldof use.

A robot may include a driving unit having an actuator or a motor toperform various physical operations such as moving joints thereof.Because a wheel, a brake, a propeller, and the like are included in adriving unit of a movable robot, the movable robot is capable of movingon the ground or flying in the air through the driving unit.

FIG. 1 is a view illustrating the appearance of a robot according to anembodiment of the present invention.

FIG. 1 illustrates a robot 100 which is a stationary robot that does notmove, but the present invention is not limited thereto and the followingdescription may also apply to mobile robots.

Referring to FIG. 1, the robot 100 includes a first body part 102 whichis a lower part, and a second body part 104 above the first body part102.

Here, the first body part 102 is fixed. The second body part 104 isrotatable on the first body part 102 to the left or right and may beangle-adjustable (i.e., tiltable) in a vertical direction.

A camera 106 is attached to an upper surface of the second body part104. Thus, the camera 106 is rotated and tilted together with therotation and tilting of the second body part 104. A focal length of thecamera 106 is adjustable and thus an image zoom function is usable.

A microphone 108, a speaker 110, and a display 112 are also attached tothe second body part 104. A sensor unit may be attached to an outer sideof the second body part 104, and a communicator, a memory, and aprocessor may be provided inside the second body part 104.

FIG. 2 is a block diagram illustrating a control relationship betweenmain components of a robot 100 according to an embodiment of the presentinvention.

Referring to FIG. 2, the robot 100 according to an embodiment of thepresent invention is an electronic device applicable to performing avideo call and includes a camera 106, a microphone 108, a speaker 110, adisplay 112, a communicator 114, a sensor unit 116, a memory 118, and aprocessor 120 as described above.

Functions of these component will be described in detail below.

The camera 106 may acquire a video of a space. Here, the space may be anindoor space or an outdoor space. As described above, as the second bodypart 104 rotates and tilts, the camera 106 may be rotated and tilted tothe left or right, and the focal length of the camera 106 may beadjusted.

The microphone 108 receives a voice signal output from the space. Asdescribed above, the microphone 108 may be a multi-channel microphone.

The speaker 110 may be used during a video call and may output to asound source signal included in a video transmitted by a counterpartrobot or terminal in the video call (hereinafter referred to as the“counterpart robot” for convenience of description) to the space.

The display 112 may include a liquid crystal display (LCD), alight-emitting diode (LED) display, an organic LED (OLED) display, orthe like, and is a device capable of displaying an image or an imageframe to a user. In particular, the display 112 may be a touch displaywith a touch device.

The communicator 114 establishes communication with the counterpartterminal. That is, the communicator 114 transmits a video created by therobot 100 to the counterpart terminal and receives a video transmittedfrom the counterpart terminal.

In this case, the communicator 114 may include a mobile communicationmodule, a short-range communication module, and the like.

The mobile communication module may transmit a radio signal to orreceive a radio signal from at least one of a base station, an externalterminal device, or a communication server in a mobile communicationnetwork established according to mobile communication technologystandards or a communication method, e.g., the Global System for Mobilecommunication (GSM), Code Division Multi-Access (CDMA), CDMA2000,Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO),Wideband CDMA (WCDMA), High-Speed Downlink Packet Access (HSDPA),High-Speed Uplink Packet Access (HSUPA), Long-Term Evolution (LTE),LTE-Advanced (LTE-A), or the like.

The short-range communication module is for short-range communicationand includes at least one of Bluetooth, Radio Frequency IDentification(RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee,Near-Field Communication (NFC), Wi-Fi, Wi-Fi Direct, or WirelessUniversal Serial Bus (USB) technology.

The sensor unit 116 may include at least one sensor and sense specificinformation regarding an external environment of the robot 100. As anexample, the sensor unit 116 may include a light detection and ranging(LiDAR) sensor, a radar sensor, an infrared sensor, an ultrasonicsensor, a radio-frequency (RF) sensor, and the like to measure adistance to an object (a user, etc.) located near the robot 100 and mayfurther include various other sensors such as a geomagnetic sensor, aninertial sensor, a photo sensor, and the like.

The memory 118 may include a volatile memory and/or a nonvolatile memoryand stores instructions or data related to at least another component ofthe terminal device 100. In particular, the memory 118 may storeinstructions or data related to a computer program or a recording mediumfor making a video call.

The processor 120 may include one or more among a central processingunit, an application processor, and a communications processor. Theprocessor 120 may execute operations or data processing related tocontrol and/or communication of at least another component of the robot100 and execute instructions related to the execution of the computerprogram.

A video communication method using the robot 100 according to anembodiment will be described in more detail with reference to FIG. 3below.

FIG. 3 is a flowchart of a video communication method using the robot100 according to a first embodiment of the present invention.

It is assumed that at the start of a video call, a face of the robot 100faces one of the plurality of users located in a space. In this case,the one user may be closest to the robot 100.

It is also assumed that when the video call is started, the robot 100receives a second video, which is a video of the other party, from acounterpart terminal (robot), displays the second video on the display112, and receives a sound source signal included in the second video.

Operations of the video communication method will be described in detailbelow.

In operation S302, the camera 106 acquires a video of a space for makinga video call. The video acquired by the robot 100 will be hereinafterreferred to as a first video.

The camera 106 may rotate to the left or right or tilt upward ordownward and thus the first video may be captured in all directions ofthe space.

In operation S304, the microphone 108 receives a sound signal outputfrom the space.

In this case, the sound signal may include a plurality of voice signalsoutput from the plurality of users located in the space.

In operation S306, the processor 120 measures positions at which theplurality of voice signals included in the sound signal were generated.In this case, the positions of generation of the plurality of voicesignals may be angles at which the plurality of voice signals weregenerated relative to a position of the robot 100.

As an example, the processor 120 may calculate positions (angles) ofgeneration of the plurality of voice signals by applying a sound sourcelocalization (SSL) algorithm to a sound signal received via amulti-channel microphone.

In operation S308, the processor 120 calculates positions of N users inthe acquired first video (here, N is an integer greater than or equal to2). That is, the processor 120 calculates the positions of the N usersby analyzing the first video and using information sensed by the sensorunit 116.

In this case, the N users in the first video are users who are within anangle of view of the camera 106 when a video is captured by the camera106 in a specific direction, i.e., users in the first video.

In operation S310, the processor 120 selects N voice signals generatedat the same positions as the N users in the first video from among aplurality of voice signals. That is, the processor 120 selects N voicesignals generated within the angle of view of the camera 106 from amongthe plurality of voice signals. The N voice signals are voice signalsoutput from the N users.

In operation S312, the processor 120 removes noise from the N voicesignals. Various noise cancellation techniques are applicable tooperation S312.

In operation S314, the processor 120 calculates a ratio of times duringwhich a voice is detected from a waveform of each of the N voice signalsin a previous time period prior to a first time point.

In this case, the first time point may be a current time point. Theprevious time period may be a specific time period prior to the firsttime point. For example, the previous time period may be 2 or 3 seconds.The ratio of the times during which the voice is detected refers to atotal time during which a user spoke in the previous time period.

In an embodiment of the present invention, the waveform of each of the Nvoice signals in the previous time period may include at least one voiceoutput section and at least one non-voice output section, and theprocessor 120 may calculate a ratio of a total time of at least onevoice output section to the total time of the previous time period. Inthis case, the calculated ratio corresponds to a ratio of times duringwhich the voice is detected from the waveforms.

Operation S314 will be described in more detail with reference to thefollowing drawings.

FIG. 4 is a diagram illustrating the concept of a voice output sectionand a non-voice output section of a user's voice signal according to anembodiment of the present invention.

More specifically, FIG. 4A shows an example of a waveform in a timedomain of a certain user's voice signal.

Referring to FIG. 4A, the waveform in the time domain of the user'svoice signal includes at least one section of strong signal intensityand at least one section of weak signal intensity. In this case, thesection of strong signal intensity is a section in which the user spoke,i.e., a voice output section, and the section of weak signal intensityis a section in which the user did not speak, i.e., a non-voice outputsection.

The processor 120 may use a predetermined threshold intensity todistinguish between the voice output section and the non-voice outputsection. That is, the processor 120 determines whether the intensity ofthe voice signal is greater than or equal to the predetermined thresholdintensity at a time point in a total period of the waveform on the basisof the waveform of the user's voice signal. In this case, the thresholdintensity may be determined experimentally, and may be, for example,very low signal intensity.

Accordingly, the processor 120 may determine a section in which theintensity of the voice signal is greater than or equal to the thresholdintensity as the voice output section, and a section in which theintensity of the voice signal is less than the threshold intensity asthe non-voice output section.

FIG. 4B is a graph including a voice output section/non-voice outputsection based on a waveform of a voice signal.

Here, the X-axis of the graph is the same as the X-axis of the waveform,and the Y-axis of the graph has a value of 0 or 1. A section with theY-axis of 0 corresponds to the non-voice output section, and a sectionwith the Y-axis of 1 corresponds to the voice output section. Thus, theprocessor 120 may calculate a total time of at least one voice outputsection using the graph and calculate a ratio of times during which avoice is detected from the waveform of the voice signal on the basis ofthe calculated total time.

Referring back to FIG. 3, in operation S316, the processor 120determines a main user of the video call at the first time point on thebasis of the calculated ratio of times. That is, in operation S316, themain user who is a main speaker of the video call among the N userswithin the angle of view of the camera 106 is determined.

In an embodiment of the present invention, the processor 120 may set afirst user corresponding to a maximum ratio of times among calculatedratios of times of the N users' voice signals as a main user (a mainspeaker) of the video call.

That is, the processor 120 may set a user who spoke most frequently,i.e., a user who spoke actively, in the previous time period prior tothe first time point as the first user. In other words, a user who spokemost frequently in the previous time period is a user who led aconversation with the other party and thus may be determined as a mainuser.

A case in which there are two or more first users who spoke mostfrequently in the previous time period may occur. In this case, therobot 100 cannot determine a main user.

In order to prevent the above case, according to an embodiment of thepresent invention, the processor 120 may determine a first user locateda minimum distance from the robot 100 as a main user.

That is, a person closest to the robot 100 is likely to be a person whois most interested in the video call. Thus, the processor 120 maydetermine, as a main user, a first user located the minimum distancefrom the robot 100 among the two or more first users.

In this case, when the sensor unit 116 is not provided or does notoperate, the processor 120 may determine the first user located theminimum distance from the robot 100 by analyzing the first video. Thatis, the processor 120 may estimate as a main user a first user whoseface looks biggest in the first video from among the two or more firstusers.

More specifically, a person whose face looks biggest in the video isestimated to be a person closest to the robot 100 due to perspective.Therefore, the processor 120 may identify the first users' faces,calculate the sizes of the identified first users' faces by calculatingthe diagonal lengths thereof, normalize the sizes of the first users'faces according to the distances between pupils, and determine a firstuser having a maximum face size among the normalized first users' facesas a main user.

Next, in operation S318, the processor 120 controls the acquiring of thefirst video of the camera 106 such that the main user is located at thecenter of the first video.

In operation S320, the processor 120 creates a first sound source signalmatching the first video. That is, the first sound source signal is asound source signal output to the counterpart robot together with thefirst video. In operation S322, the communicator 114 transmits the firstsound source signal together with the first video to the counterpartrobot under control of the processor 120. In this case, the processor120 may create the first sound source signal to include the N voicesignals, excluding the remaining voice signals except the N voicesignals among the plurality of voice signals.

That is, in order to improve the quality of the video call, it ispreferable to remove a voice signal of a user who does not appear in thefirst video, i.e., a voice signal of a user who is not located withinthe angle of view of the camera 106. Accordingly, the processor 120 maycalculate positions at which the plurality of voice signals included inthe sound signal were generated, divide the plurality of voice signalsinto N voice signals within the angle of view of the camera 106 andremaining voice signals beyond the angle of view of the camera 106, andcreate the first sound source signal to include only the N voicesignals.

To sum up, according to the present invention, the quality of the videocall may be improved by determining the main user of the video call andremoving voice signals not present in the first video on the basis ofthe above description.

FIG. 5 is a flowchart of a video communication method using a robot 100according to a second embodiment of the present invention.

Operations of the video communication method will be described in detailbelow.

First, in operation S502, N voice signals matching positions of N usersappearing in a first video are selected, and noise is removed from the Nvoice signals. Operation S502 is the same as operations S302 to S312 ofFIG. 3 described above and thus a detailed description thereof will beomitted here.

In operation S504, the processor 120 calculates a ratio of times duringwhich a voice is detected from a waveform of each of N voice signals ina first previous time period prior to a first time point, and a ratio oftimes during which the voice is detected from a waveform of each of Nvoice signals in a second previous time period prior to the first timepoint. Operation S504 may be substantially the same as operation S314 ofFIG. 3 described above.

Here, the first previous time period and the second previous time periodare previous specific time periods prior to the first time point, andthe first previous time period is longer than the second previous timeperiod. The first previous time period is a previous time period used todetermine a user who spoke actively in a relatively long time period,and the second previous time period is used to determine a user whospoke actively in a relatively short time period.

FIG. 6 illustrates the concept of a first previous time period and asecond previous time period. When a first time point is a time point of“2 seconds” on the X-axis, a first previous time period may be a sectionhaving a duration of 1.5 seconds from the first time point, and a secondprevious time period may be a section having a duration of 0.5 secondsfrom the first time point.

Referring back to FIG. 5, in operation S506, the processor 120 selects asecond user corresponding to a maximum ratio of times in the firstprevious time period from among N users on the basis of the calculatedratio of times in the first previous time period. In operation S508, theprocessor 120 selects a third user corresponding to a maximum ratio oftimes in the second previous time period from among the N users on thebasis of the calculated ratio of times in the second previous timeperiod.

Thereafter, in operation S510, the processor 120 determines whether thesecond user and the third user are identical.

When the second user and the third user are identical, in operationS512, the processor 120 determines the second user as a main user. Next,in operation S514, the processor 120 controls the acquiring of the firstvideo of the camera 106 such that the second user is located at thecenter of the first video.

That is, a situation in which the second user and the third user are thesame refers to a situation in which one user spoke actively both in along time period and a short time period. Therefore, the camera 106 maycapture the first video such that the second user is positioned at thecenter of the angle of view.

Conversely, when the second user and the third user are not identical,in operation S516, the processor 120 determines the third user as a mainuser. In this case, in operation S518, the processor 120 controls theacquiring of the first video of the camera 106 such that the third useris positioned at the center of the first video for a predetermined timeand then the second user is positioned at the center of the first video.

That is, a situation in which the second user and the third user are notidentical is a situation in which the second user spoke actively in along time period but the third user suddenly spoke a lot at a morerecent time point than the first time point. Accordingly, the camera 106may acquire the first video such that the third user is located at thecenter of the first video for a short time and thereafter the seconduser is located at the center of the first video.

When there are two or more second users, in operation S518, a seconduser whose face looks biggest in the first video or a second userlocated a minimum distance from the robot 100 may be selected. Whenthere are two or more third users, in operation S510, a third user whoseface looks biggest in the first video or a third user located a minimumdistance from the robot 100 may be selected. When one of two or morethird users is identical to the second user, in operation S512, it isdetermined that the second user and the third user are identical.

In operation S520, the processor 120 creates a first sound source signalmatching the first video. In operation S522, the communicator 114transmits the first sound source signal along with the first video tothe counterpart robot under control of the processor 120.

FIG. 7 is a flowchart of a video communication method performed betweena robot 100 and a counterpart robot 700 according to a third embodimentof the present invention.

In operation S702, communication is established between the robot 100and the counterpart robot 700. Thus, the robot 100 may transmit a videoto or receive a video from the counterpart robot 700 in real time.

In operation S704, the robot 100 receives a specific touch event fromthe user.

As an example, the touch event may be swiping a touch display by a userin one of up, down, left, and right directions. As another example, thetouch event may be increasing or decreasing the distance between theuser's two fingers touching the touch display.

In operation S706, the robot 100 creates an operation control commandfor the counterpart robot 700 on the basis of the received touch event.

As an example, when the touch event is a swiping motion, the operationcontrol command may be a command to adjust an angle of the counterpartrobot 700 in a direction in which swiping is performed.

As another example, when the touch event is increasing the distancebetween fingers, the operation control command may be a command tocontrol a focal length for a zoom-in operation of a camera included inthe counterpart robot 700.

As another example, when the touch event is decreasing the distancebetween fingers, the operation control command may be a command tocontrol a focal length for a zoom-out operation of the camera includedin the counterpart robot 700.

In operation S708, the robot 100 transmits the operation control commandto the counterpart robot 700. In operation S710, an operation of thecounterpart robot 700 may be controlled on the basis of the motioncontrol command.

That is, according to the present invention, a user may experience thefun of a video call through interactions between robots.

When the operation of the counterpart robot 700 is controlled on thebasis of the operation control command, a case in which M users among Nusers within an angle of view of the camera of the counterpart robot 700are moving beyond the angle of view, i.e., the M users are disappearingin a video, may occur (here, M is an integer greater than or equal to1).

In this case, a processor of the counterpart robot 700 may create asecond sound source signal by reducing the intensity of M voice signalsoutput from the M users and transmit the second sound source signal tothe robot 100. That is, in order to deliver an effect of disappearanceto a counterpart user, the counterpart robot 700 may reduce theintensity of the disappearing users' voice.

According to an embodiment of the present invention, the processor ofthe counterpart robot 700 may reduce the intensity of the M voicesignals by using the differences in angles between a center axis of thecamera and the positions of the M users. For example, the differences inangle between the center axis of the camera and a disappearing user maybe inversely proportional to the intensities of the signals to bereduced.

The intensity of the reduced voice signals may be lower than that of alowest voice signal of a user appearing in the video. Accordingly, theeffect of disappearance can be clearly transmitted to the counterpartuser.

According to the present invention, even when a video call is conductedby a plurality of users, the quality of the video call can be improved.

The effects of the present invention are not limited to the aboveeffects, and various other effects of the present invention may beeasily derived from the configuration of the present invention by thoseof ordinary skill in the art.

Although it is described that all components of an embodiment of thepresent invention are combined with each other or are operated whilebeing combined with each other, the present invention is not necessarilylimited thereto, and at least one of all the components may be operatedwhile being selectively combined with each other without departing fromthe scope of the present invention. Although each of all the componentsmay be embodied as independent hardware, some or all of the componentsmay be selectively combined to realize a computer program having aprogram module which performs some or all of functions of a combinationof one or more hardware units. Code and code segments constituting thecomputer program can be easily reasoned by those of ordinary skill inthe art. The computer program may be stored in a computer-readablemedium, and an embodiment of the present invention may be implemented byreading and executing the computer program. Examples of thecomputer-readable medium storing the computer program include a magneticrecording medium, an optical recording medium, and a storage medium witha semiconductor recording element. The computer program for implementingthe present invention includes a program module transmitted in real timevia an external device.

While embodiments of the present invention have been particularlydescribed, various changes or modifications may be made therein bygeneral technical experts. It is therefore to be understood that suchchanges and modifications are included within the scope of the presentinvention unless they depart from the scope of the present invention.

The present invention is directed to providing a video communicationmethod of increasing the quality of a video call when the video call ismade by a plurality of users, and a robot for implementing the same.

Aspects of the present invention are not limited thereto, and otheraspects and advantages of the present invention which are not mentionedherein will be apparent from the following description and will be moreclearly understood by embodiments of the present invention. It will alsobe readily understood that aspects and advantages of the invention canbe implemented by means defined in the appended claims and a combinationthereof.

According to an aspect of the present invention, a robot includes acamera configured to acquire a first video of a space for a video call;a multi-channel microphone configured to receive a sound signal outputto the space; a memory storing one or more instructions; and a processorconfigured to execute the one or more instructions to perform operationsfor determining a first user among N users in the first video based onthe sound signal received in a previous time period prior to a firsttime point, wherein the first user is a main user of the video call atthe first time point and N is an integer greater than or equal to 2.

According to another aspect of the present invention, a videocommunication method, performed by a robot, includes acquiring, by acamera, a first video of a space for a video call; receiving, by amulti-channel microphone, a sound signal output to the space; anddetermining, by a processor, a first user based on the sound signalreceived in a previous time period prior to a first time point, whereinthe first user is a main user of the video call at the first time pointand N is an integer greater than or equal to 2.

Any reference in this specification to “one embodiment,” “anembodiment,” “example embodiment,” etc., means that a particularfeature, structure, or characteristic described in connection with theembodiment is included in at least one embodiment of the invention. Theappearances of such phrases in various places in the specification arenot necessarily all referring to the same embodiment. Further, when aparticular feature, structure, or characteristic is described inconnection with any embodiment, it is submitted that it is within thepurview of one skilled in the art to effect such feature, structure, orcharacteristic in connection with other ones of the embodiments.

Although embodiments have been described with reference to a number ofillustrative embodiments thereof, it should be understood that numerousother modifications and embodiments can be devised by those skilled inthe art that will fall within the spirit and scope of the principles ofthis disclosure. More particularly, various variations and modificationsare possible in the component parts and/or arrangements of the subjectcombination arrangement within the scope of the disclosure, the drawingsand the appended claims. In addition to variations and modifications inthe component parts and/or arrangements, alternative uses will also beapparent to those skilled in the art.

What is claimed is:
 1. A robot comprising: a camera configured toacquire a first video of a space for a video call; a multi-channelmicrophone configured to receive a sound signal output to the space; amemory storing one or more instructions; and a processor configured toexecute the one or more instructions to perform operations for:determining a first user among N users in the first video based on thesound signal received in a previous time period prior to a first timepoint, wherein the first user is a main user of the video call at thefirst time point and N is an integer greater than or equal to
 2. 2. Therobot of claim 1, wherein the first user is a user who spoke mostfrequently in the previous time period prior to the first time point. 3.The robot of claim 1, the first user is a user whose voice outputsection is maximum in the previous time period prior to the first timepoint, the voice output section is a section in which the user spoke inthe previous time interval of the first time point.
 4. The robot ofclaim 1, wherein the camera acquires the first video such that an imageof a first user is provided at a center of frames of the first video atthe first time point.
 5. The robot of claim 1, wherein, when two or morefirst users are determined as the main user, the processor determines,from the first video, the main user as the first user whose face in thefirst video appears to be larger, or the processor determines, from thefirst video, the main user as the first user who is located a minimumdistance from the robot.
 6. The robot of claim 1, further comprising: acommunication device configured to communicate with a counterpartterminal, and wherein, the processor is to provide a first sound sourcesignal corresponding to the first video based on the sound signal, andthe communication device transmits, to the counterpart terminal, thefirst sound source signal along with the first video, wherein, aplurality of users are located in the space, voice signals of the Nusers are included in the first sound source signal, and voice signalsof one or more users other than the N users of the plurality of users isnot included in the first sound source signal.
 7. The robot of claim 1,further comprising: a communication device configured to communicatewith a counterpart terminal, and wherein, the processor is to provide asecond sound source signal corresponding to the first video based on thesound signal, and the communication device transmits, to the counterpartterminal, the second sound source signal along with the first video,wherein, the N users include a first group of user and a second group ofuser, the first group of user is in the first video, the second group ofuser is removed from the first video, wherein, the second sound sourcesignal includes a voice signal of the first group of user whoseintensity is maintained and a voice signal of the second group of userwhose intensity is reduced.
 8. The robot of claim 1, further comprising:a communication device configured to receive a second video for thevideo call, the second video being transmitted from a counterpartterminal; and a touch display to display the second video, wherein, acommunication device transmits an operation control command of thecounterpart terminal generated based on a touch event input to the touchdisplay by a user, and an operation of the counterpart terminal iscontrolled by the operation control command.
 9. The robot of claim 1,wherein, the sound signal includes a plurality of voice signals, theprocessor selects voice signals output by the N users from the pluralityof voice signals based on the first video and the sound signal anddetermine a ratio of times during which a voice is detected from awaveforms of each of the voice signals output by the N users in aprevious time period prior to a first time point, and the first user isa user who outputs a voice signal having a maximum ratio of time.
 10. Avideo communication method of a robot, the method comprising: acquiring,by a camera, a first video of a space for a video call; receiving, by amulti-channel microphone, a sound signal output to the space; anddetermining, by a processor, a first user based on the sound signalreceived in a previous time period prior to a first time point, whereinthe first user is a main user of the video call at the first time pointand N is an integer greater than or equal to 2.